Click any tag below to further narrow down your results
Links
This arXiv paper proposes Latent Sparse Attention (LSA), which clusters keys in a learned latent space so queries score cluster hierarchies rather than every key, aiming to make attention cheap at over 1M tokens. The authors report speedups over full and original sparse attention at 10M tokens while keeping accuracy close to the baseline.
- Full attention becomes too costly above 1M tokens, and existing sparse indexers either remain quadratic or lose accuracy at that scale.
- LSA's distilled variant (D-LSA) speeds up one attention layer at 10M tokens by 22.7x in prefill and 4.6x in decode over the original sparse attention, with similar accuracy.
- The native variant (N-LSA), trained only on next-token loss, matches full attention accuracy with a 1.6x per-layer training speedup at 128k tokens.
David Breunig's talk on how context engineering emerged after the long-context push of 2024, why bigger context windows introduced new failure modes, and the six patterns builders now use to manage what goes into a model's context. It's a useful map of where the practice has been and where it's headed.
- Million-token context windows didn't solve the problem. Long contexts add room for errors as well as information, leading to failure modes like context poisoning, distraction, confusion, and clash.
- Gemini 2.5 playing Pokémon shows poisoning in practice: a fact baked into its weights (the tea bribe from the older game) got stuck in its context and cost it hours. Performance also degraded past roughly 100,000 tokens, with the agent repeating old actions instead of planning new ones.
- Breunig's research cites shows instruction-following drops as bad examples pile up in context, affecting every model tested, including smaller ones that fall below half adherence after a few contradictions. The six-tool toolbox (augmentation, loadout, quarantine, pruning, offloading, summarization) has stayed stable as these problems emerged.
@0xMortyx on X
A data scientist ran a blind test pitting RAG against Kimi K3's full 1M-token context window on the same 12 questions. Long context scored perfectly on answer quality but cost about 16 times as much and ran roughly 3 times slower, so the post argues the right choice depends on corpus size and query volume.
- Long context answered all 12 questions perfectly, while RAG was near-perfect but weaker on completeness.
- Long context cost about 16 times as much and took roughly 3 times as long per question; at 12,000 queries the cost gap reaches about $3,800 versus $230.
- Skipping RAG is only backed by evidence when the corpus sits well under a fifth of the window; the test never covered corpora filling 60-80% of it.
- For tasks spanning many unconnected sources, the author points to Kimi K3's Agent Swarm and a connected context graph instead of either approach.
@monospodcast on X
A Google arXiv paper proposes replacing full conversation history in long AI agent sessions with a structured representation of current state, reportedly cutting token usage by 94%. The author, from the Monos podcast account, calls it potentially the best news for AI economics this year, at least for users.
- The method swaps the entire conversation history for a structured snapshot of the agent's current state.
- The paper reports a 94% reduction in token usage during long agent sessions.
- The author argues this matters most for users paying per-token costs, since long agent runs get expensive fast.
OpenAI is rolling out a new model called GPT-5.2-Codex-Max for subscribers, which enhances the capabilities of its Codex tool. This version improves performance on long tasks, tool use reliability, and understanding of visual content, building on the features introduced in GPT-5.2. Further details about the model are expected to be released soon.
- OpenAI is rolling out GPT-5.2-Codex-Max, a new Codex model, to some subscribers
- It improves performance on long tasks, tool use reliability, and visual content understanding compared to GPT-5.2
- More details on the model are expected to be released soon